[JSC] Module linking: import slots without sorting; nothing resolved to compare records until there are two - #665

Since #522 a module environment has an import slot per import binding, and records of the same source in different loaders can share a ModuleProgramExecutable. Three parts of that ran for every record of every loader, whether or not any other record ever shared its code:

• importSlotNames() collected, atomized and sorted the record's import local names. The slot count is needed to create the module environment, so every record paid for it at instantiation, and importSlotIndex() binary-searched the sorted names by string at each import access site when a CodeBlock links — after resolveImport() had already looked the same import up.
• getOrMakeExecutable() resolved every import of the record to build the bindings that executables are compared by.
• evaluate() resolved every import again to fill the slots before running the module.

Before #522 an import was resolved when the code that uses it was first linked, so modules with many imports that are not all used while the program starts got slower to load.

Changes

Slots are numbered without comparing names.

• A prelinked record's slot is the import's position among the graph's imports of the module. That array is already in a fixed order; importSlotCount() is its length. A namespace import's position is a slot nothing uses.
• Any other record's slot is its position among the import entries, which iterate in insertion order, with a name → index map for lookups. No sort, no codePointCompare.
• ImportedBinding now carries the slot, so two records only share an executable when they number every binding the same way. (Without that, a prelinked record and an ordinary record of the same source could have equal bindings and different numbering, because only the prelinked one leaves a gap at a namespace import.)

One lookup per import access site. JSScope::abstractAccess calls JSModuleRecord::resolveImportWithSlot, which for a prelinked record finds the import in the graph once and gets both the resolution and the slot from it. tryResolveImportPrelinked gains a by-entry overload next to the by-name one, the same pair tryResolveExportPrelinked has; the by-name form finds the entry and calls the other.

Nothing is resolved to compare records until there are two to compare. getOrMakeExecutable() used to resolve every import of every record to build the bindings executables are compared by, whether or not a second record for the module ever turned up. Now the executable remembers the record it was linked for (weakly) and is registered as before; when another record for the same key, source and module scope arrives, JSModuleRecord::resolvesImportsLike() computes the first record's bindings then, once, and keeps them on the executable (the record is not needed after that, so it may be collected while others go on sharing). A record that is the only one to link a module resolves nothing for this. If the first record is collected before a second arrives there is nothing to compare with; the second links its own code and later ones share with it.

A prelinked record lists only the imports its graph does not answer. An import the graph resolved to a binding of another of its modules is left out of importedBindings() when the record itself resolves it to that binding of that module's record — judged by tryResolveImportPrelinked's answer, which is kept and is what the record's code links against, so it holds whether the record reaches the module through the loader's table of the graph's records or, once that entry has been removed, through the edge pinned to it. That is the same variable of the same source for every record of that graph module that resolves it so; the executable notes which graph module it was linked for, and only records of that same module are compared with it. What is left to compare is what the graph could not answer (builtins, modules outside the graph, star exports it could not flatten) and anything a record resolved some other way, which it lists and so differs; a prelinked and an ordinary record of one source never share.

evaluate() fills slots up front only for an executable that is shared. Code only one record runs fills them on first use: LLInt and the baseline/LOL slow paths call fillImportSlot, and DFG already treats an empty slot as "exit and let the slow path fill it". The second and later records know they share when they link, before they evaluate.

Tests

JSTests/modules/import-slots-*.js (26 files, fixtures in JSTests/modules/import-slots/) and JSTests/wasm/modules/js-wasm-cycle-loaders.js:

• On the global object's own loader, where slots now fill on first use: every kind of export; names whose source, code point and hash orders all differ, string and non-BMP names; 128 bindings in one module; namespace imports before, between and after named ones, and modules with only namespace imports or none; live bindings through renaming, star, chained and namespace re-exports and aliases; imports never read, or first read long after evaluation, or first read on a path only taken once the function is hot; cycles with TDZ and hoisted functions, a module importing itself; top-level await; aliases, shadowing, names of globals; reads from arrows, generators, async functions, eval, default parameters, class static blocks, accessors and the rest; every form of assignment to an import; default-export forms; dynamic import(); link errors.
• Across additional loaders: a module the own loader loaded, and ran hot, before any other loader existed is shared with loaders made later, as is one it loads afterwards, in either order; several loaders and the own loader starting on the same modules at once, so that records are compared with one whose body (suspended at a top-level await) has yet to run; a record that links alone and only evaluates after a second one has taken its code and made it hot; the only instance of a module collected, with its code, before another loader asks for it; an exporter that is not a source text module (a JSON module: the binding's offset is what is compared); a JS↔wasm cycle loaded from either root, where from the wasm root the JS module links while the wasm module has no environment yet, links code of its own, and later records are compared with that; modules that fail to link, in every loader, each time; sharing for each namespace/named mix, for 128 bindings, for cycles, self-imports and top-level await, with each instance keeping its own bindings in every tier; an instance that is the first to read an import in code another instance made hot; instances collected while others keep running.

They pass under default options, low tier-up thresholds with --useConcurrentJIT=false, the same with FTL, --useJIT=false, --useLLInt=false and --useDFGJIT=false.

Also: every other test in JSTests/modules (including module-loaders*.js) passes under default options and under low thresholds. Bun built against this branch passes its suites for programs built with bun build --compile with and without --bytecode (prelinked and ordinary records) and with and without splitting, and for prelinked records in several loaders of one global object.

Timing

Bun built against main and against this branch, same machine, interleaved runs, medians. A generated program of 400 modules, about 40 import bindings each of which a sixth are read while the program starts:

• run from source: 79.4 → 75.3 ms (45 runs each);
• built with bun build --compile --bytecode --splitting, every module a chunk of its own: 22.6 → 19.6 ms (60 runs each).